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Preface 


This  project  comes  as  the  result  of  many  years  of  flying  and  watching  others  fly  both 
high  performance  and  transport  aircraft.  It  is  based  upon  intimate  knowledge  of  both  the 
potential  for  human  error  and  the  saving  grace  associated  with  multiple  perspectives.  It 
also  fully  acknowledges  and  welcomes  the  vast  capabilities  associated  with  increasing 
levels  of  automation  in  aircraft  cockpits.  Pilots  continually  play  the  game  of  “what  if’, 
envisioning  possible  futures  and  situations  and  planning  resolutions,  given  the  resources 
available.  Operational  decision-makers  must  continually  do  the  same  based  upon  a 
realistic  understanding  of  possible  mission  scenarios  and  the  capabilities  and  limitations 
of  the  aircraft  and  crew  that  will  fly  them. 
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Abstract 

The  newest  airlifter  in  the  Air  Force  inventory,  the  C-17,  has  a  cockpit  crew 
complement  of  two  (pilot  and  copilot).  This  is  a  departure  from  the  traditional  airlift 
crew  makeup  that  included  additional  crewmembers  to  handle  navigation  and  aircraft 
systems.  The  extensive  use  of  cockpit  automation  allowed  this  crew  reduction.  Major 
airlines,  also  making  extensive  use  of  cockpit  automation,  are  finding  challenges  with 
automation  in  their  fairly  routine  and  mundane  ops  (compared  with  that  of  the  military). 
Recent  research  has  shown  that  automation  can  actually,  rather  than  decreasing  workload, 
increase  cockpit  workload  beyond  that  of  less  automated  aircraft  during  those  periods 
where  workload  is  already  traditionally  high.  The  mission  of  the  C-17  is  likely  to  see  a 
less  experienced  crew  flying  low-level  in  a  threat  environment,  talking  to  multiple 
controlling  agencies,  and  trying  to  get  into  a  location  they've  never  seen  after  a  15  hour 
double  air-refueling  flight  from  the  continental  United  States.  These  issues  warrant 
operational  consideration  when  employing  the  C-17  on  some  of  it’s  most  demanding 
missions,  and  are  also  applicable  to  other  new  or  upgraded  aircraft  where  similar 
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cockpit/crew  situations  exist. 


Part  1 


To  Err  Is  Human 


At  2142  eastern  standard  time  (est),  on  December  20,  1995,  American 
Airlines  Flight  965  (AA965),  a  Boeing  757-223,  N651AA,  on  a  regularly 
scheduled  passenger  flight  from  Miami  International  Airport,  Florida, 
U.S.A.,  to  Alfonso  Bonilla  Aragon  International  Airport,  in  Cali, 
Colombia,  operating  under  instrument  flight  rules  (IFR),  crashed  into 
mountainous  terrain  during  a  descent  from  cruise  altitude  in  visual 
meteorological  conditions  (VMC).  The  accident  site  was  near  the  town  of 
Buga,  33  miles  northeast  of  the  Cali  VOR.  The  airplane  impacted  at  about 
8,900  feet  mean  sea  level  (msl),  near  the  summit  of  El  Deluvio  and 
approximately  10  miles  east  of  Airway  W3.  Of  the  155  passengers,  2 
flightcrew  members,  and  6  cabincrew  members  on  board,  4  passengers 
survived  the  accident.  There  was  no  evidence  of  failures  or  malfunctions 
in  the  airplane,  its  components,  or  its  systems.  Weather  was  not  a  factor  in 
this  accident. 


—  AA965  -  Cali  Accident  Report 


Without  a  doubt,  the  only  certainty  associated  with  human  performance  is  error.  The 
degree  to  which  this  inherent  error  influences  the  desired  outcome  determines  the  relative 
success  of  the  endeavor.  Since  the  concept  of  error  is  central  to  the  discussion  to  follow, 
it  should  be  defined  as  it  applies  to  aviation.  For  the  purposes  of  this  paper,  I  will  define 
error  as;  “the  action  or  inaction  that  leads  to  a  deviation  from  crew  or  organizational 
intentions  or  expectations.”1 
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A  Quick  Look  Back  to  Cali 


The  captain  of  AA965  had  flown  into  Cali  thirteen  times  without  incident.  Both  of 
AA965's  pilots  were  experienced  in  the  airplane  and  proficient  in  the  use  of  it’s 
automated  systems,  as  noted  by  their  peers.2 

The  B-757  is  a  modern,  highly  automated  aircraft  that  uses  a  Flight  Management 
System  (FMS)  and  integrated  flight  management  computer  (FMC).  This  is  the  “heart”  of 
the  automated  cockpit  and  contains  a  worldwide  navigation  data  base  and  performance 
data  which,  combined  with  pilot  inputs,  govern  autothrottle  and  autopilot  functions.  The 
FMS  monitors  the  system  and  engine  status  and  displays  the  information,  as  well  as 
airplane  attitude,  flightpath,  navigation,  and  other  information,  through  cathode  ray  tube 
(CRT)  displays.  Pilot  inputs  into  the  FMS  can  be  performed  either  through  a  keyboard 
and  associated  CRT,  or  to  a  more  limited  extent  via  controls  on  the  glareshield 

On  December  20,  1995,  most  likely  because  of  the  self-induced  time  pressure  and  an 
attempt  to  execute  the  approach  without  adequate  preparation,  the  crew  of  AA965 
committed  a  critical  error  by  executing  a  change  of  course  through  the  FMS  without 
verifying  its  effect  on  the  flightpath.  The  evidence  indicates  that  either  the  captain  or  the 
first  officer  selected  and  executed  a  direct  course  to  the  identifier  "R,"  in  the  mistaken 
belief  that  R  was  Rozo  (the  intended  fix)  as  it  was  identified  on  the  approach  chart.  The 
pilots  could  not  know  without  verification  with  the  cockpit  navigation  displays  or 
considerable  calculation  that  instead  of  selecting  Rozo,  they  had  selected  the  Romeo 
beacon,  located  near  Bogota,  some  132  miles  east-northeast  of  Cali.  In  executing  a  turn 
toward  Romeo  rather  than  Rozo,  the  crew  had  the  airplane  turn  away  from  Cali  and 
towards  mountainous  terrain  to  the  east  of  the  approach  course,  while  the  descent 
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continued.  At  this  time,  both  pilots  also  attempted  to  determine  the  airplane's  position  in 


relation  to  the  initial  approach  fix  (IAF).  Neither  pilot  was  able  to  determine  why  the 
navaid  was  not  where  they  believed  it  should  be,  and  neither  noted  nor  commented  on  the 
continued  descent.  The  cockpit  voice  recorder  (CVR)  indicates  that  the  flightcrew 
became  confused  and  attempted  to  determine  their  position  through  the  FMS.  For 
example,  at  2138:49  the  first  officer  asked,  "Uh,  where  are  we?"  and  again,  9  seconds 
later  asked,  "Where  [are]  we  headed?"  The  captain  responded,  "I  don't  know  ...  what 
happened  here?"  The  discussion  continued  as  each  attempted  to  determine  the  position 
and  path  of  the  airplane  relative  to  the  approach  to  Cali.  Less  than  three  minutes  later, 
the  aircraft  impacted  the  ground. 

The  probable  causes  listed  in  the  investigation  are: 

1.  The  flightcrew's  failure  to  adequately  plan  and  execute  the  approach  to  runway 
19  at  Cali  and  their  inadequate  use  of  automation. 

2.  Failure  of  the  flightcrew  to  discontinue  the  approach  into  Cali,  despite  numerous 
cues  alerting  them  of  the  inadvisability  of  continuing  the  approach. 

3.  The  lack  of  situational  awareness  of  the  flightcrew  regarding  vertical  navigation, 
proximity  to  terrain,  and  the  relative  location  of  critical  radio  aids. 

4.  Failure  of  the  flightcrew  to  revert  to  basic  radio  navigation  at  the  time  when  the 
FMS-assisted  navigation  became  confusing  and  demanded  an  excessive 
workload  in  a  critical  phase  of  the  flight. 

This  accident  was  not  caused  by  the  automated  systems  on  AA965.  Many 
opportunities  existed  for  a  break  in  the  chain  of  events  that  led  to  their  controlled  flight 
into  terrain.  There  were,  however,  automation  issues  that  played  a  significant  role  in  the 
decision  making  that  took  place  that  night  with  that  crew. 
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Notes 


1  Robert  L.  Helmreich,  James  R.  Klinect,  &  John  A.  Wilhelm,  Models  of  Threat, 
Error,  and  CRM  in  Flight  Operations ,  (Presented  at  the  10th  International  Symposium  on 
Aviation  Psychology  Columbus,  Ohio:  May  3-6,  1999). 

Aircraft  Accident  Report.  American  Airlines  Flight  965.  Boeing  757-223,  N651AA. 
Near  Cali,  Colombia ,  Aeronautica  Civil  Of  The  Republic  of  Colombia,  20  Dec  1995. 
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Part  2 


Setting  The  Stage 

Real  knowledge  is  to  know  the  extent  of  one's  ignorance. 

—  Confucius 


Automation  Issues  -  A  Short  Primer 

In  the  early  1980s,  the  aviation  industry  began  to  embrace  the  new  capabilities  made 
possible  with  the  growth  in  computer  technology.  Automated  systems  and  computers 
began  to  take  the  place  of  traditional  systems.  As  the  automated  systems  and  levels  of 
information  available  to  pilots  grew,  accidents  and  incidents  related  to  the  new  systems 
also  grew,  as  would  be  expected.  The  increased  automation  also  made  possible  smaller 
crew  sizes  because  of  the  ability  to  automate  the  systems  traditionally  monitored  and 
controlled  by  dedicated  navigators  and  flight  engineers.  This  is  the  environment  in  which 
the  United  States  Air  Force  (USAF)  and  the  prime  contractor  McDonnell  Douglas 
designed  and  built  the  C-17. 

C-17  Automation  Philosophy 

With  the  capabilities  now  available  because  of  automation,  the  C-17  was  designed  to 
operate  with  a  cockpit  crew  of  only  two  pilots.  The  crew  of  the  aircraft  it  was  designed 
to  replace,  the  C-141,  had  included  as  part  of  the  crew  both  a  navigator  and  flight 
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engineer.  Although  borrowing  heavily  from  the  equipment  in  the  automated  cockpits  of 
its  civilian  transport  counterparts,  some  basic  cockpit  design  philosophies  were 
significantly  different  with  C-17.  Bill  Casey,  Chief  Pilot  for  McDonnell  Douglas, 
described  the  differences  as  “instead  of  automated  high  technology,  like  the  commercial 
world,  we  used  the  Lincoln  Logs,  or  a  simplified,  “dumb”  cockpit  approach.  If  you  don’t 
know  what  it  does,  don’t  touch  it,  and  it  won’t  do  anything.”1  This  philosophy  was 
designed  to  allow  for  the  lower  experience  level  and  more  demanding  missions  of  the 
average  military  pilot,  compared  to  his/her  civilian  counterpart.  A  tradeoff  in  this 
decision  is  that  “an  expert  in  the  C-17  cockpit  will  have  to  push  more  buttons  to 
command  a  specific  action  than  his  airline  counterpart.”- 

The  automation  philosophy  outlined  in  the  directive  that  governs  flight  operations  for 
the  C-17  supports  this  by  saying  pilots  should  use  appropriate  levels  of  automation  as 
required  by  the  flight  conditions  -  their  first  priority  is  to  fly  the  aircraft.  It  goes  on  to  say 
that  “The  Automatic  Flight  Control  System  (AFCS)  and  Mission  Computer  (MC)  are 
intended  to  aid  in  workload  management,  not  complicate  it.  As  the  flight  situation 
changes,  [pilots  should]  not  feel  locked  into  a  level  of  automation.” 

Major  Automation  Issues 

Under  a  grant  from  the  US  Federal  Aviation  Administration,  a  team  of  researchers 
from  Oregon  State  University  and  Research  Integrations,  Incorporated  recently  conducted 
a  study  on  human  factors  issues  of  commercial  transport  aircraft  flight  deck  automation 
that  drew  from  many  sources.4  This  study  surveyed  aviation  experts,  reviewed  literature 
and  accident  reports,  and  analyzed  incident  reports  to  compile  data  and  other  objective 
evidence  related  to  flight  deck  automation  issues.  In  the  study,  they  compiled  a  list  of 
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flight  deck  automation  human  factors  issues  and  related  evidence.  They  rank  ordered 
these  issues  in  several  ways  based  upon  data  found  during  surveys,  literature  reviews,  and 
incident  and  accident  analysis.  Based  upon  the  multiple  rankings  and  criteria  across  the 
span  of  data  collected,  they  also  developed  a  meta-ranking  of  the  issues.5  The  top  five 
issues  (from  Table  3,  Appendix  A)  related  to  flight  deck  automation  in  this  meta-ranking 
were: 

1.  Automation  may  demand  attention. 

2.  Automation  behavior  may  be  unexpected  and  unexplained. 

3.  Pilots  may  be  overconfident  in  automation. 

4.  Failure  assessment  may  be  difficult. 

5.  Behavior  of  automation  may  not  be  apparent. 

Monitoring  and  Workload 

Associated  with  cockpit  automation  and  error  is  the  concept  of  monitoring. 
Automated  systems  must  be  continually  monitored  to  ensure  safe  operation.  Independent 
studies  by  both  the  International  Civil  Aviation  Organization  (ICAO)  and  the  National 
Transportation  Safety  Board  (NTSB)  show  that  inadequate  monitoring  has  contributed  to 
a  large  number  of  the  accidents.6 

Another  related  issue  is  that  of  workload.  Automated  systems  are  designed  to 
decrease  workload  through  automation  of  control,  navigation  and  information  systems. 
Studies  are  now  indicating  however,  that  automation  may  not  always  work  as  designed. 
During  periods  where  workload  is  traditionally  low,  such  as  during  cruise,  workload  is 
decreased  and  pilots  simply  monitor  the  systems.  During  periods  of  traditionally  high 
workload,  however,  automation  can  actually  increase  crew  activity,  such  as  during 
departures  and  arrivals.  Even  small  increases  in  workload  during  these  time-critical 
phases  of  flight  can  have  serious  implications. 
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Who  Gets  the  Job  Done? 

There  are  some  important  points  to  address  here.  Certain  levels  of  automated 
systems  must  be  used  to  operate  and  navigate  modern  aircraft.  The  roles  and 
responsibilities  of  the  crew  positions  that  were  eliminated  (in  the  case  of  the  C-17,  the 
navigator  and  flight  engineer)  must  still  be  accomplished,  either  through  the  use  of  on¬ 
board  automated  systems  or  manually  by  the  pilots.  Reversion  to  manual  flight  and 
navigation  in  lieu  of  using  the  AFCS  and  MC  adds  further  to  the  workload  in  the  cockpit, 
although  it  may  decrease  the  cognitive  issues  associated  with  trying  to  cope  with  an 
automated  system  which  is  not  fully  understood  or  is  functioning  in  a  confusing  way. 

Crew  Resource  Management  (CRM)  and  It’s  Role 

In  the  early  1980s,  programs  were  started  with  the  intent  of  reducing  accidents  that 
came  as  a  result  of  pilot  error.  The  training  programs  focused  on  group  dynamics  and 
awareness.  Scenarios  were  used  to  examine  how  situations  can  develop  in  which  it  is 
possible  to  “break  the  chain”  of  events  that  could  culminate  in  a  compromise  in  safety 
through  leadership,  group  skills,  and  communication.  CRM  has  been  developed  and 
refined  over  the  years  but  continues  to  be  a  very  important  part  of  the  training  that  Air 
Force  and  civil  aircrew  receive.  Today,  strategies  and  techniques  for  dealing  with 
automation  are  central  issues  in  CRM  training.  Mission  oriented  simulator  training  and 
regular  simulator  and  in-flight  training  using  varying  degrees  of  automation  ensure 
familiarity  with  automated  systems,  proficiency  in  dealing  with  simulated  emergencies, 
and  continued  honing  of  manual  skills.  CRM  is  the  primary  line  of  defense  against  the 
threats  to  safety  that  are  present  in  aviation  and  inherent  in  human  performance. 
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The  Threat  and  Error  Management  Model 


It  is  important  to  understand  that  cockpit  automation  can  impact  the  workload  on  the 
flight  deck,  and  ultimately  the  potential  safety  of  the  crew.  Even  more  important  from  an 
operator  and  operating  organization  standpoint  is  trying  to  understand  how  this  impact 
occurs,  and  how  best  to  deal  with  it.  One  model  recently  introduced  by  researchers  at  the 
University  of  Texas  at  Austin  is  the  Threat  and  Error  Management  Model  (TEMM). 
Through  extensive  participation  in  Line  Operations  Safety  Audits  (LOSA)  of  carriers,  the 
group  developed  a  general  model  of  threat  and  error  in  aviation.  Beginning  with  both 
external  and  internal  sources  of  threat  and  applying  CRM  behaviors  to  recognize  and 
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avoid  or  detect  and  manage,  several  outcomes  are  possible.  Figure  1  is  a  graphic 
depiction  of  this  basic  model. 


External 

Threats 


Internal 

Threats 


CRM 

Behaviors 


Outcomes 


Expected 

Events/ 

Risks 


Unexpected 
Events/ 
Risks 


External 

Error 


Threat 
Recognition 
and  Error 
Avoidance 
Behaviors 


/ 


A  Safe 
Fiight 


Error 

Detection  L 
and 

Management 

Behaviors 


Recovery  to 
a  Safe 
Fiight 


Additionai 

Error 


Incident/ 

Accident 


Figure  1  The  Model  of  Flightcrew  Error  Management  9 
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Using  this  model,  a  structure  exists  that  can  be  used  to  examine  a  particular  incident 
or  a  notional  mission.  In  this  case,  it  will  be  used  to  frame  the  mission  issues  that  a  C-17 
crew  is  likely  to  encounter. 


C-17  Mission  Description 

The  C-17  Aircraft 

As  described  in  the  Flight  Manual:  The  C-17  is  a  long-range,  air-refuelable,  heavy 
logistic  transport  aircraft.  It’s  design  characteristics  give  it  the  capability  to  operate  into 
and  out  of  short  runways  and  austere  airfields  carrying  large  payloads.  The  engine 
thrust  reverser  system  is  capable  of  backing  a  fully  loaded  aircraft.  The  pilots  are 
provided  with  Head  Up  Displays  (HUD),  four  Multi-Function  Displays  (MFD)  and 
redundant  mission  computers  which  reduce  pilot  workload  and  enhance  mission 
capability.  Station  Keeping  Equipment  (SKE)  is  provided  for  multiship  formation  flights 
and  eardrop  in  instrument  meteorologiccd  conditions  (IMC).  This  widebody  aircraft  is 
designed  to  airlift  palletized  cargo,  rolling  stock,  troops,  passengers,  and  aeromedical 
evacuation  patients.  An  aeried  delivery  system  provides  the  capability  to  airdrop  troops 
or  a  variety  of  materiel  and  equipment  systems.10 

The  Mission 

The  mission  capabilities  of  the  C-17  are  significantly  different  from  modem  civilian 
transports.  For  the  purposes  of  this  paper,  I  will  describe  the  mission  tasks  rather  than 
look  at  a  particular  mission. 

A  final  consideration  related  to  the  mission  of  the  C-17  is  that  of  flight  duty  periods. 
Flight  duty  period,  according  to  AFI  11-202,  Volume  3,  is  the  period  that  starts  when  an 
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aircrew  reports  for  a  mission,  briefing,  or  other  official  duty  and  ends  when  engines  are 
shut  down  at  the  end  of  a  mission.1 1  The  standard  flight  duty  period  for  a  two-pilot  crew 
in  the  C-17  is  16  hours.  During  contingency  operations,  it  is  not  uncommon  to  have  this 
standard  flight  duty  period  extended  by  an  hour  or  two  based  upon  the  nature  of  the 
operation.  With  an  additional  pilot,  a  crew  may  be  considered  “augmented”  and  have 

13 

their  allowable  flight  duty  period  extended  to  24  hours. 
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Part  3 


C-17  Mission  Analysis  Using  the  TEMM  Model 


You  may  have  to  fight  a  battle  more  than  once  to  win  it. 

—  Margaret  Thatcher 

The  external  and  internal  threats  identified  in  the  following  sections  are  simply  some 
of  those  suggested  by  experience  and  recent  research.  The  identified  items  in  no  way 
comprise  a  comprehensive  list,  but  they  do  provide  a  good  basis  to  frame  a  discussion  of 
some  of  the  issues  at  hand. 


External  Threats 


Table  1  Potential  External  Threats  In  C-17  Mission 


Expected  Events/Risks 

Unexpected  Events/Risks 

Terrain/Low-Level  Flight 

Mission  Change 

Language  Barriers 

Mechanical  Problems 

Procedural  Change 

Missed  Communications 

Multiple  Controlling  Agencies 

Unexpected  Air  Traffic 

Formation  Flight 

Controlling  Agency  Error 

Surface-to-Air/Air-to-Air  Threats 

Unexpected  Weather 

Poor/Incorrect  Intelligence  Information 

Pressure  to  Accomplish  Mission 

External  threats  are  those  situations,  events,  or  errors  that  originate  from  outside  the 
cockpit.1  They  are  inherent  in  both  military  and  civilian  flying,  and  have  been  since 
aviation  began.  Unique  challenges  exist  for  the  military  pilots  because  of  the  nature  of 
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the  environments  in  which  they  fly.  Table  1  gives  examples  of  some  of  the  expected  and 
unexpected  risks  that  could  be  encountered  on  an  operational  C-17  mission.  Expected 
risks  may  be  thought  of  as  those  routine  risks  that  are  not  outside  the  expectations  for  a 
routine  mission,  while  unexpected  risks  are  those  outside  the  bounds  of  the  “normal 
mission.”  Training  scenarios  and  mission  oriented  simulators  incorporate  a  variety  of 
these  same  threats  in  order  to  practice  procedures  and  review  tactics  and  techniques  for 
dealing  with  any  one  or  a  combination  of  such  external  threats.  Automated  systems,  such 
as  the  aircraft  defensive  systems,  are  also  present  that  aid  in  detection  and  resolution  of 
some  of  the  threats. 


Internal  Threats 

Unlike  the  external  threats,  I  will  focus  on  several  specific  internal  threats  because 
they  are  inherent  (to  some  extent)  in  any  crew,  and  typically  exacerbate  the  effect  of 
external  threats. 


Table  2  Potential  Internal  Threats  In  C-17  Mission 


_ Internal  Threats _ 

Unfamiliarity  With  Location/Destination 
Fatigue/Mission  Duration 

Challenging  Flight  Maneuvers _ 

Crew  Inexperience 

Intentional  Noncompliance _ 

Automation  Interface  Issues 


Unfamiliarity  With  Location/Destination 

Unfamiliarity  with  an  area  or  destination  is  not  an  unusual  aspect  of  an  operational 
airlift  or  airdrop  mission.  In  fact  it  is  entirely  possible  that  prior  to  a  mission,  a  pilot  will 
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not  have  heard  of  the  destination  country ,  let  alone  the  destination  airfield.  In  1997,  air 
mobility  forces  transited  all  but  five  of  the  world’s  countries,  three  of  which  had  no 
airfield.2 

Air  Mobility  Command  (AMC),  the  controlling  major  command  for  the  USAF 
mobility  forces,  strives  to  provide  airfield  information  to  aircrews.  All  airfields  that  are 
likely  to  be  transited  by  USAF  aircraft  have  been  surveyed  for  suitability.  This  database 
is  maintained  in  the  form  of  an  Airfield  Suitability  and  Restrictions  Report  (ASRR), 
which  contains  specific  runway  and  taxiway  information  on  an  airfield,  along  with 
restrictions  such  as  daylight  operations  only.  Some  more  routinely  transited  airfields 
have  videotapes  on  file  that  visually  take  the  viewer  down  an  approach  to  the  various 
mnways  at  the  field,  and  discuss  terrain  and  airfield  hazards.  Airfields  that  are 
considered  particularly  hazardous  are  often  designated  as  certification  airfields,  meaning 
that  before  flying  into  that  airfield/region  in  command  of  a  mission,  the  pilot  must  have 
flown  there  with  another  pilot  familiar  with  the  area. 

Fatigue/Mission  Duration 

Fatigue  has  significant  implications  concerning  error.  The  crew  duty  periods 
regularly  encountered  by  mobility  aircrew  can  create  challenges  due  to  fatigue.  A  NASA 
Technical  Memorandum  (TM)  on  the  issue  of  civilian  airline  crew  duty  periods  indicated 
that  scientific  findings  from  a  variety  of  sources,  including  data  from  aviation, 
demonstrate  a  significantly  increased  vulnerability  for  performance-impairing  fatigue 
after  12-hours.  Another  aspect  of  this  issue  is  when  missions  are  flown.  From  personal 
experience,  we  know  that  performance  drops  during  the  normal  nighttime  sleep  period 
and  that  it  is  more  difficult  to  sleep  during  the  day.  Research  has  often  confirmed  these 
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commonly  known  facts  as  they  relate  to  aviation.  A  NASA  TM  concludes  that  "circadian 
disruption  can  lead  to  acute  sleep  deficits,  cumulative  sleep  loss,  decreases  in 
performance  and  alertness,  and  various  health  problems."  A  study  of  naval  watch 
keepers  found  that  between  0400  to  0600,  response  rates  drop  33  percent,  and  response 
speed  eight  percent,  compared  with  rates  between  2000  to  2200  hours.4 

The  NASA  memorandum  recommends  that  flights  and  missions  take  "circadian 
stability"  into  account.  NASA  goes  on  to  state  that  nighttime  flying  imposes  different 
physiological  challenges  as  compared  to  daytime  flying.  These  differences  must  be 
accommodated  in  the  timing  and  duration  of  rest  periods  and  would  also  require  limiting 
the  number  of  consecutive  nights  of  flying.5 

Challenging  Flight  Maneuvers 

The  nature  of  military  flying  and  the  unpredictable  and  sometimes  hostile 
environment  which  they  train  for  and  fly  in  often  requires  challenging  flight  maneuvers 
designed  to  limit  exposure  to  other  external  threats.  Some  of  these  maneuvers  include 
formation  flight  with  multiple  aircraft  occupying  a  relatively  small  airspace,  extended 
low-level  flight  requiring  constant  maneuvering  both  to  avoid  terrain  and  use  it  for  threat 
avoidance,  specially  designed  threat  avoidance  approach  and  departure  procedures  which 
involve  greater  maneuvering  and  climb/descent  rates  than  normally  used,  and 
landings/takeoffs  on  short/austere  unimproved  runways.  The  C-17  is  designed  to  be 
capable  of  flight  maneuvers  which  allow  it  to  operate  in  this  environment.  These 
capabilities  increase  workload  and  themselves  may  become  threat  sources.  Commercial 
flying  has  fewer  of  these  threat  sources,  but  a  statistical  summary  of  accidents  for  the 
worldwide  commercial  jet  fleet  between  1988  thru  1997  indicate  that  the  flight 
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maneuvers  associated  with  takeoff,  initial  climb,  final  approach,  and  landing  account  for 
68  percent  of  the  accidents,  while  only  constituting  6  percent  of  the  flight  time.6  These 
flight  segments  are  typically  involve  both  vertical  and  horizontal  maneuvering  near 
terrain.  The  autopilot  is  typically  off  for  a  portion  of  this  flight  segment,  but  interaction 
with  automated  systems  for  navigation,  aircraft  configuration  and  situational  awareness  is 
at  a  peak  level.  It  is  also  the  period  where  modes  are  being  changed  to  and  from 
automated  flight.  It  is  within  these  flight  segments  where  cockpit  workload  is  the 
highest,  and  where  the  need  to  monitor  and  interact  with  automated  systems  may  further 
increase  workload  as  discussed  earlier. 

Crew  Inexperience 

The  experience  of  military  pilots  varies  widely,  but  compared  to  their  civilian 
counterparts  at  the  major  airlines,  the  military  pilots  are  relatively  inexperienced. 
Military  airlift  pilots  typically  upgrade  to  aircraft  commanders  when  they  meet  the 
minimum  number  of  flying  hours  required.  For  the  C-17,  this  can  be  as  low  as  1300 
hours  (total  USAF  flying  and  simulator  time  -  not  all  in  the  C-17).  200  hours  later,  the 

pilot  may  upgrade  to  the  instructor  pilot  position.  In  contrast,  to  be  hired  as  a  copilot  by 
Southwest  Airlines  requires  a  minimum  of  2500  hours  total  or  1500  hours  in  a  turbine 
aircraft  to  including  a  minimum  of  1000  hours  in  turbine  aircraft  as  the  Pilot  in  Command 
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(specifically  excludes  any  simulator  time).  The  total  number  of  flight  hours  logged  by 
the  military  pilot  do  not  alone  pose  a  threat;  however,  experience  and  “airmanship”  are 
still  being  developed  (due  to  the  lower  experience  level)  to  a  much  greater  degree.  This 
lower  experience  level  can  result  in  steeper  learning  curves,  fewer  “real  life”  experiences 
to  draw  on,  and  less  maturity.  The  military  counters  this  with  a  much  more  robust  and 


16 


comprehensive  training  program  than  is  provided  to  the  civilian  airline  pilots.  This 
cannot  however,  fully  substitute  for  real-world  experience. 

Intentional  Noncompliance 

A  recent  University  of  Texas  study  found  after  observing  184  commercial  flight 
crews  during  Line  Operations  Safety  Audits  that  the  most  frequently  committed  (and  also 
least  consequential)  errors  by  the  crews  were  intentional  noncompliance.9  These  include 
willful  violations  of  established  procedures,  such  as  failure  to  reference  checklists. 
Anecdotal  evidence  also  suggests  that  the  tendency  to  deviate  from  procedure  and  skip 
standard  operating  procedure  (such  as  checklist  usage)  increases  during  periods  of  high 
workload  where  time  and  excess  capacity  become  limited. 

Automation  Interface  Issues 

The  previous  internal  threats  addressed  have  been  inherent  in  military  flying  for 
many  years.  Part  two  outlined  several  automation  issues  associated  with  modem  aircraft. 
Through  the  use  of  automation,  the  USAF  has  decreased  the  minimum  cockpit  crew  size 
to  two  pilots  in  the  case  of  the  C-17.  Cockpit  automation  offers  significant  benefits  and 
capabilities;  however,  it  also  brings  with  it  human  factors  issues  and  potential  threats  that 
are  very  real.  The  scope  and  implications  of  these  threats  are  still  being  studied,  and  are 
far  from  being  fully  understood.  When  faced  with  an  emergency,  USAF  pilots  are  taught 
from  the  very  start  of  their  careers  to  do  four  things: 

1.  Maintain  Aircraft  Control 

2.  Analyze  the  situation 

3.  Take  Appropriate  Actions 

4.  Land  as  Soon  as  Conditions  Permit 
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This  process  is  now  inextricably  linked  with  computer  systems  and  automation.  In  1988, 
McDonnell  Douglas  conducted  an  aircrew  workload  evaluation  using  a  full  mission 
scenario  (as  envisioned  at  the  time).  One  of  the  purposes  of  this  initial  evaluation  was  to 
establish  confidence  that  aircrew  workload  was  within  acceptable  limits  during  ideal 
mission  conditions  (no  unexpected  external  threats).10  This  evaluation  found  no  periods 
during  which  workload  was  classified  as  very  high  or  greater;  however,  long  periods  (30 
minutes  during  low  altitude  cruise)  were  found  in  which  the  workload  allowed  very  little 
spare  capacity,  but  maintenance  of  effort  in  the  primary  tasks  not  in  question.11  This  lack 
of  spare  capacity  in  the  automated  two  person  cockpit,  coupled  with  the  addition  of 
unexpected  external  threats  and  some  of  the  automation  issues  described  in  Part  2  may  be 
significant. 


Where  It  Comes  Together 

As  shown  in  the  TEMM  model,  the  front  line  for  recognizing  and  avoiding  or 
detecting  and  managing  threats  is  the  crew.  The  model  describes  this  as  CRM  behaviors, 
but  it  comes  down  to  people.  Operationally,  the  C-17  is  required  to  have  a  minimum 
crew  of  two  pilots.  Missions  are  frequently  flown  with  additional  pilots  due  to  crew  ratio 
(number  of  crew  to  number  of  available  aircraft)  and  training  requirements.  A  third  pilot 
can  drastically  decrease  the  workload  on  a  crew  by  performing  basic  tasks  and  acting  as 
an  observer.  As  an  observer,  they  can  often  maintain  a  greater  sense  of  situational 
awareness,  and  provide  input  as  needed  to  increase  the  capacity  of  the  crew  as  a  whole. 
During  periods  of  high  workload,  a  third  pilot  is  an  extra  set  of  eyes  and  ears  that  can 
take  on  more  tasks  to  decrease  the  load  on  the  pilots  that  occupy  the  primary  crew 
positions.  When  missions  are  frequently  flown  with  additional  pilots,  negative  training 
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can  occur  as  the  crews  naturally  become  accustomed  to  delegating  tasks  and  having  an 
extra  hand  available  during  periods  of  high  workload.  The  impact  of  this  can  be  a  crew 
unaccustomed  to  working  without  an  extra  hand  if  operational  requirements  push  crew 
size  back  to  the  basic  two-pilot  crew. 

We  have  discussed  how  individuals  are  sources  of  error,  but  they  are  also  the  last 
defense  against  an  error  leading  to  an  incident  or  accident.  Their  effective  use  of  CRM  in 
managing  all  of  the  resources  at  their  disposal  to  safely  accomplish  a  mission  is  the 
desired  goal. 
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Part  4 


Conclusions 

You  live  and  learn.  Or  you  don  7  live  long. 

—  Lazarus  Long1 

The  TEMM  model  (figure  1)  depicted  earlier  provides  a  good  structure  with  which  to 
view  the  operational  hazards  associated  with  mobility  operations  in  the  C-17,  the 
demands  on  and  sources  of  error  within  a  crew,  and  the  ultimate  responsibility  levied  on 
pilots. 

Error  is  inherent  in  everything  that  we  do.  In  aviation,  technical  proficiency, 
established  organizational  procedures,  the  ability  to  maximize  the  benefits  of  working  as 
a  crew,  and  the  ability  to  work  as  a  crew  to  recognize  and  mitigate  both  internal  and 
external  errors  are  key  to  safe  and  effective  mission  accomplishment.  Realistic  training 
with  an  emphasis  on  crew  coordination  and  use  of  automation  can  greatly  improve 
effectiveness  and  safety.  It  is  the  responsibility  of  operational  organizations  to  conduct 
planning  and  training  with  an  understanding  of  this. 

Military  transport  and  tanker  aircraft  operate  on  a  routine  basis  with  generally  less 
experienced  aircrews  and  flying  more  demanding  mission  than  their  civilian  counterparts. 
At  the  extreme,  they  operate  in  a  hostile  environment.  Automated  systems  can  and  have 
vastly  increase  the  capability  of  military  transport  and  tanker  aircraft,  but  studies  are  also 
showing  that  cockpit  automation  can  also  negatively  affect  the  performance  during  the 
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times  when  it  can  least  be  afforded.  In  the  case  of  the  C-17  the  cockpit  is  designed  in 
such  a  way  that  maximum  use  of  automation  is  more  demanding  than  that  of  large  civil 
aircraft.  Presumably,  this  may  also  apply  in  the  case  of  other  military  aircraft  that  are 
being  planned,  built,  or  upgraded  using  highly  automated  systems. 

Initial  workload  evaluations  conducted  to  validate  the  crew  complement  indicated 
that  portions  of  the  envisioned  mission,  specifically  the  descent,  low  altitude  cruise,  and 
approach/landing,  had  workloads  that  allowed  very  little  spare  capacity.  Many 
modifications  have  been  made  to  C-17  aircraft  systems  from  those  for  which  that 
evaluation  was  conducted,  but  during  that  time  we  have  also  learned  more  about  the 
potential  for  automation  to  add  to  workload. 

Addition  of  a  third  crewmember  creates  a  significant  synergistic  effect  in  the  ability 
of  a  crew  to  deal  with  mission  demands.  The  routine  (but  informal)  addition  of  a  third 
crewmember  with  no  formal  plan  to  ensure  their  presence  during  demanding  missions 
could  result  in  a  crew  that  is  less  prepared  to  operate  under  those  critical  conditions. 
Addition  of  a  third  crewmember  significantly  bolsters  the  last  line  of  defense  against  both 
internal  and  external  threats,  and  can  greatly  enhance  the  safety  and  capability  of  a  crew 
to  deal  with  an  uncertain  environment.  With  these  considerations  in  mind,  a  formal 
evaluation  of  the  potential  benefits  of  a  three-person  cockpit  for  those  missions  that  are 
the  most  demanding  may  be  warranted.  In  any  event,  leaders  making  operational 
decisions  related  to  employment  of  the  C-17  must  be  aware  of  the  issues  associated  with 
modem,  highly  automated  aircraft,  as  well  as  the  crew  limitations  and  potential  for 
human  error  that  have  been  part  of  aviation  since  its  earliest  days. 
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Notes 


1  Robert  A.  Heinlein,  The  Notebooks  of  Lazarus  Long  (Rohnert  Park,  CA: 
Pomegranate  Artbooks,  1995). 
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Appendix  A 


Flight  Deck  Automation  Issues 


Table  3  Top  20  Flight  Deck  Automation  Issues  (Multiple  Criteria)  1 


Meta  Ranking 

Abbreviated  Issue  Statement 
(Study  Issue  ID) 

Ranking  (Out  of  92) 

Sum  of  Rankings 

By  Citation 

By  Expert 
Agreement 

By  Expert 
Criticality 

By  Sum  of 
Strengths 

i 

Automation  May  Demand  Attention  (102) 

i 

2 

10 

18 

31 

2 

Automation  Behavior  May  Be  Unexpected  And  Unexplained 
(108) 

3 

23 

18 

8 

52 

3 

Pilots  May  Be  Overconfident  In  Automation  (131) 

2 

32 

23 

5 

62 

4 

Failure  Assessment  May  Be  Difficult  (025) 

16 

6 

17 

26 

65 

5 

Behavior  Of  Automation  May  Not  Be  Apparent  (083) 

7 

20 

34 

6 

67 

6 

Mode  Transitions  May  Be  Uncommanded  (044) 

25 

4 

11 

31 

71 

7 

Mode  Awareness  May  Be  Lacking  (095) 

11 

54 

3 

10 

78 

8 

Mode  Selection  May  Be  Incorrect  (145) 

33 

21 

13 

16 

83 

9 

Situation  Awareness  May  Be  Reduced  (114) 

17 

50 

6 

12 

85 

10 

Understanding  Of  Automation  May  Be  Inadequate  (105) 

4 

57 

35 

1 

97 

11 

Fluman-Centered  Design  Philosophy  May  Be  Lacking  (100) 

32 

15 

12 

39 

98 

12 

Training  May  Be  Inadequate  (133) 

5 

46 

45 

3 

99 

13 

Crew  Assignment  May  Be  Inappropriate  (142) 

49 

33 

20 

30 

102 

14 

Automation  May  Not  Work  Well  Under  Unusual  Conditions 
(150) 

28 

47 

28 

15 

104 

15 

Pilots  May  Over-Rely  On  Automation  (106) 

15 

62 

39 

4 

105 

16 

Pilots  May  Be  Out  Of  The  Loop  (002) 

18 

8 

5 

22 

107 

17 

Database  May  Be  Erroneous  Or  Incomplete  (110) 

30 

37 

44 

32 

114 

18 

Manual  Skills  May  Be  Lost  (065) 

6 

61 

64 

9 

116 

19 

Automation  May  Be  Too  Complex  (040) 

13 

51 

37 

11 

122 

20 

Interface  May  Be  Poorly  Designed  (039) 

10 

11 

49 

13 

123 

Notes 

1  Ken  Funk,  Beth  Lyall,  and  Candy  Suroteguh.  Flight  Deck  Automation  Issues. 
Issues  Ranked  by  Multiple  Criteria  ( Meta-Ranking );  on-line,  Internet,  22  March  2000, 
available  from  http://flightdcck.ie.orst.edu/FDAI/Mcta- 

Analysis/meta(by_Meta_Ranking).html. 
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Appendix  B 


Accidents/Fatalities  by  Phase  of  Flight 


Accidents  and  Onboard  Fatalities  by  Phase  of  Flight 

Hull  Loss  and/or  Fatal  Accidents  — Worldwide  Commercial  Jet  Fleet  —  1988  through  1997 
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**  Cruise  includes  4  accidents  (84  fatalities)  with  unknown  flight  phase. 


20 


1987  STATISTICAL  SUMMARY.  JUNE  IW8 


maeioKar 


Figure  2.  Accidents/Fatalities  By  Phase  of  Flight  1988-19971 


Notes 

1  Boeing  Commercial  Airplane  Group  (1998).  Statistical  Summary  of  Commercial 
Jet  Aircraft  Accidents:  Worldwide  Operations,  1959-1997.  Published  annually  by 
Airplane  Safety  Engineering,  pg.  20. 
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Fatalities 


Glossary 


AFCS 

Automatic  Flight  Control  System 

AMC 

Air  Mobility  Command 

ASRS 

Aviation  Safety  Reporting  System 

CRM 

Cockpit/Crew  Resource  Management 

CRT 

Cathode  Ray  Tube 

CVR 

Cockpit  Voice  Recorded 

FMC 

Flight  Management  Computer 

FMS 

Flight  Management  System 

HUD 

Heads  Up  Display 

ICAO 

International  Civil  Aviation  Organization 

IMC 

Instrument  Meteorological  Conditions 

IFR 

Instrument  Flight  Rules 

LOSA 

Fine  Operational  Safety  Audit 

MC 

Mission  Computer 

MFD 

Multi-Function  Display 

NTSB 

National  Transportation  Safety  Board 

SKE 

Station  Keeping  Equipment 

TEMM 

Threat  and  Error  Management  Model 

TM 

Technical  Memorandum 

USAF 

United  States  Air  Force 

VFR 

Visual  Flight  Rules 

VMC 

Visual  Meteorological  Conditions 
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